DOCUMENT RESUME 



ED 263 787 



FL 015 412 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



Liskin-Gasparro, Judith E, 

Foreign Language Oral Proficiency Testing: An 

Organizing Principle for Instruction, 

84 

18p.; Paper presented at the annual meeting of the 
National Council on Measurement in Education (New 
Orleans, LA, 1984). 

Reports - Descriptive (141) ~ Speeches/Conference 
Papers (150) 

MFOl/PCOl Plus Postage. 

Conversational Language Courses; ^Curriculum Design; 
Educational Objectives; Higher Education; 
^Interviews; ^Language Proficiency; ^Language Tests; 
Program Evaluation; ^Rating Scales; Secondary 
Education; Second Language Instruction; *Speech 
Communication; Student Evaluation; Test Use 
*ACTFL ETS Proficiency Guidelines 



The oral proficiency interview, a one-to-one 
conversational interview structured to elicit the best performance 
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them can serve as an organizing principle lor curriculum development 
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have been found to be more effective than other instructional 
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Since 1981 the foreign language teaching profession has been 
actively engaged in exploring the ramifications of oral proficiency 
assessment — the testing of students' ability to use the spoken language 
in real-life settings — for methodology, curriculum, materials, and 
classroom testing* We are discovering that proficiency testing provides 
a valuable perspective on the instructional process that was absent 
before; we simply had no way of knowing, except anecdotally, how well 
students'. classroom learning served them in real-life situations, such 
as travel abroad or employment* The oral proficiency interview fcan 
serve as a much-needed complement to teacher-made and standardized 
achievement tests. In addition, the rating scale used to assess student 
performance provides a basis for curriculum planning and the elaboration 
of end-of-course goals. 

In this paper, I will give some background on the oral proficiency 
interview — what it is, where it came from, and how it operates — and then 
will describe the ways in which it is being used for student assessment, 
program assessment, and curriculum development. 

THE ORAL PROFICIENCY INTERVIEW; AN INTRODUCTION 

The oral proficiency interview is a structured one-on-one 
conversational interview that can last anywhere from five to 20 minutes. 
The resulting speech sample is then rated on a scale from Novice (some 
knowledge of words and phrases, but no practical ability to function in 
the language) to Superior (professional proficiency in the language). 
All of the ratings except Superior, I.e. Novice, Intermediate, and 
Advanced, have been subdivided to provide for greater discrimination. 
Even so, the ratings are properly thought of as ranges, rather than 
points on a scale, since the description of proficiency at each level 
and even sub-level is broad enough to include weaker and stronger 
performances over a significantly wide span. 
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The ratings reflect general oral ability, and the oral proficiency 
Interview differs from most other foreign language tests In th£t it Is 
not a discrete-point test of structure or vocabulary or fluency or 
pronunciation. All of these linguistic factors have been Incorporated 
Into the descriptions at each level. The ultimate criterion against 
which each speech sample la compared Is the performance of an educated 
native speaker of the language. The characteristics of speakers at each 
level can be summarized as follows: 

Novice ; Limited to memorized material. No 

practical ability to communicate. 
Usually answers with Isolated words or 
phrases. 

Intermediate : Can survive for a day or two In a 

country In which the language Is 
spoken in the way a tourist might. 
Creates with language. Can ask and 
answer questions on familiar topics 
and carry on short conversations. Can 
get Into, through, and out of a simple 
survival or courtesy situation, such 
as getting a hotel room, making a 
purchase, or Issuing an Invitation to 
a friend. Must be Intelligible to a 
native speaker used to dealing with 
foreigners attempting to speak the 
language. 

Advanced: Able to participate fully in casual 

conversations. Can narrate and describe in 
past, present, and future time, joining 
sentences together in paragraph-length 
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discourse. Can talk about personal 
history, Interests, activities, and plans. 
In addition, can report on or ('.escribe 
events In the external world. Can get 
Into, through, and out of a survival 
situation with a complication. 
Intelligible to native speakers not used to 
dealing with foreigners. 

Superior ; Sufficient control of the language to 

converse in formal and informal contexts, 
resolve problem situations, deal with 
unfamiliar topics, explain and describe in 
detail, offer supported opinions, and 
hypothesize. Can talk about a wide range 
of practical, social, professional, and 
abstract topics, particular interests, and 
special fields of competence. Errors never 
interfere with understanding and rarely 
disturb the native speaker. 

The oral interview is structured so that it elicits from the student 
the very best performance that he or she is capable of sustaining. This 
is accomplished by a four-phase conversational interview that takes the 
student from very easy "warm-up" conversation to a conversation at the 
level the student can handle comfortably. This is the solid "linguistic 
floor" of sustained performance. The tester then raises the level of 
conversation one level higher to find the "linguistic ceiling," the 
point at which the student ^8 language is not sufficient to respond to 
the linguistic demands suggested by the tester. The interview then ends 
with a gentle wind-down at a level that the student can handle 
comfortably so that he or she leaves the test with a feeling of 
accomplishment . 
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The structure of the interview and the ability of the trained tester 
to move skillfully between levels results in a revealing sample of oral 
ability. In a very short period of time the interviewer is able to see 
it all — the solid performance at a level the student can handle, and the 
breakdown in language when the tasks are too difficult. 

History 

The rating scale and the oral interview test were developed some 30 
years ago by the Foreign Service Institute of the Department of State. 
Although the descriptions of each level were originally designed for 
State Department purposes, they are general enough to apply to the 
evaluation of functional language ability in the academic context as 
well. 

The academic community became interested in the potential use of the 
interview and scale for the assessment of students' and teachers' 
foreign language oral proficiency over a decade ago, largely through the 
use of the oral interview to certify the linguistic competence of Peace 
Corps volunteers to carry out the jobs to which they had been assigned • 
Until very recently, however, the use of the oral proficiency interview 
outside the government was quite limited. 

The recent explosion of interest in proficiency assessment in 
"mainstream" academic institutions at the secondary and post-secondary 
level was sparked by the Report of President Carter's Commission on 
Foreign Language and International Studies (1979). Among its 
recommendations was one that addressed directly the issue of language 
proficiency. The Commission recommended the establishment of "a 
National Criteria and Assessment Program" that would "establish language 
proficiency achievement goals for the end of each year of study, with 
special attention to speaking proficiency," and would, in addition, 
develop tests to "assess the proficiency of both students and teachers 
in existing as well as new or experimental foreign language programs."^ 
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Although no such National Criteria and Assessment Program has been 
formally established, the American Council on the Teaching of Foreign 
Languages (ACTFL) , the largest national association devoted to foreign 
language pedagogy, and Educational Testing Service (ETS) have assumed a 
leadership role in soliciting public and private funding to carry out 
the recommendations of the President's Commission. Since 1979, grants 
to ETS, ACTFL, and other institutions and agencies have resulted in the 
following major projects: 

(1) The adaptation by ETS of the government rating scale to make it 
more usable for the assessment of traditional high school and college 
language learners. The principal modifications were the development of 
sub-ranges at the lower end of the scale and a new descriptive 
nomenclature in place of the government numerical levels. 

(2) The development by ACTFL of proficiency scales in six languages 
for listening, speaking, reading, writing and culture, based on the 
modified scale developed by ETS. These have been written as ^ first 
step in the development of articulated end-of-course goals. 

(3) The training of some 200 oral proficiency testers in eight 
languages, and of 20 of these as trainers of ethers. 

(4) One major proficiency-based curriculum development institute 
for secondary school teachers, which has resulted in the creation of a 
number of innovative projects in curriculum, materials, and assessment. 

(5) A model proficiency-based teacher certification testing 
program, developed in response to legislation in the state of Texas. 
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IMPLICATIONS FOR INSTRUCTION 

When ETS and ACTFL began training testers early in 1982 to measure 
students' oral proficiency according to the newly developed ACTFL 
Proficiency Guidelines, our sights were limited. We imagined that our 
workshop participants would use proficiency assessment for placement 
purposes, perhaps as a part of a proficiency-based language requirement 
and, looking to the future, possibly as a certification of students' 
oral skills as they left the university for the world of work. We 
frankly had not yet thought through the possible effects of proficiency 
testing on the curriculum. When these first workshop participants 
returned to their institutions after the AJ^-day intensive training 
workshop, almost immediately the reports began coming in that things 
looked different to them in the classroom. Remarks like "I never knew 
before what they could do,*' and "We've been studying the present 
subjunctive for a month, but nobody can use it in a conversational 
setting," and "Now I understand why students can't discuss themes and 
concepts in my intro to lit course; they can read at the Superior level, 
but their ability to discuss is only at the Advanced level" were common. 
More important, the workshop participants began to devise ways to 
reorganize their courses so that the curriculinn, the classroom 
activities, the materials, and the pace of instruction were all 
consciously arranged to help students cross the major thresholds between 
one proficiency level and the next. 

It has long been recognized that tests are a driving force in 
instruction. If students are to tfi;ke a particular type of standardized 
test at the end of a course or program of studies, teachers will 
understandably begin to "teach to the test," to modify their curricula 
so that students will learn what they need to know in order to perform 
well. When the test in question is unrelated to the goals and methods 
that the teacher believes in, then "teaching to the test" can be an 
educational disaster. If, on the other hand, students take proficiency 
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tests at the end of their sequence of language courses 9 tests that will 
assess hew well they can handle the language in real-life situations, 
then the pressure on the teacher to "teach to the test' can be a 
positive force in courses oriented towards performance. Programs 
oriented toward real-life language use should also include the use of 
periodic proficiency tests, both to assess both student performance and 
to evaluate program effectiveness. 

Of the instruction-related projects that have grown out of the 
ACTFL/ETS oral proficiency projects, I would like to describe two that 
illustrate the ways in which the proficiency scale and the oral 
proficiency interview are changing curricula in foreign languc-je 
programs at the grass-roots level. The two projects are (1) the 
re-structuring of a French program at the secondary-school level, based 
on the ACTFL Proficiency Guidelines; and (2) the creation of a 
fourth-year college conversation course designed to move students from 
the Intermediate level to the Advanced level. 

The pioneering work of using the ACTFL Guidelines as an organizing 
principle for curriculum development began in the summer of 1983, when 
ACTFL held an NEH-sponsored three-week summer institute for secondary 
school teachers on this topic. In the first week, the participants were 
trained to conduct and rate oral proficiency interviews and became 
immersed In the speaking guidelines. They spent weeks 2 and 3 with 
their textbooks, their curriculum guides, and the ACTFL Guidelines, 
working in small groups to develop projects to revise the curriculum or 
make classroom activities more communicative. 

These projects were tried out and refined during the fail semester 
and have been published by ACTFL as a case book. Most of the projects 
involved classroom activities, and were focused on moving students over 
a critical threshold on the proficiency scale. A second class of 
projects involved using the proficiency guidelines as starting points 
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for the development of curricula. In this regard, it is important to 
remember that the ACTFL Guidelines are not curriculum outlines, nor are 
they prescriptions for what grammatical structures to teach and when. 
They are a graduated sequence of proficiency stages around which a 
foreign language program may be structured. The "meat and potatoes" of 
that program, the day-to-day activities that constitute the sequence of 
small steps in the context of the larger phases identified by the 
guidelines, as well as the methods to be used, are still and always the 
province of the foreign language teacher. 

Perhaps the most ambitious curriculum-development undertaking to 

emerge from the ACTFL summer institute is a new set of curriculum 

guidelines for the French program at Walpole (MA) High School, designed 

by teachers Floy Miller and Charlotte Cole (1983). In the introduction 

to the first version of the curriculum outlines for French I, French II, 

and French II Honors, Miller and Cole explain that they wished to escape 

from the futile exercise that curriculum writing so often represents: 

"... we did not wish to produce curriculums that had no value for the 

classroom or ones that had little use other than cosmetic for the 

shelves or files where such documents seem inevitably to rest in peace 

2 

until the next revision." 

Miller and Cole began by examining the departmental goals for the 
language program a$ a whole, and interpreting them in light of the ACTFL 
Guidelines. They then devised end-of-year goals for speaking, 
listening, reading, writing, and culture for all five course levels in 
French. The end-of -course goals are also expressed in terms of the 
Guidelines, following a sequence that will bring students who complete 
French V to a level of proficiency consonant with the overall program 
goals set by the department. The next step was to analyze the text used 
in each course to see how the material in each unit contributed to the 
students' attainment of the end-of-course proficiency goals. Each unit 
was outlined, and day-to-day communicative "implementation" activities 

« 
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were designed to make the textbook serve more fully the goals of the 
course* 

For illustration, the proposed French I curriculum guidelines are 
printed in the handout. The function statements are taken from the 
ACTFL Guidelines (note that students completing a first-year course are 
expected to be at the Novice Level in speaking, listening, and writing, 
and at the Intermediate Level in reading); the linguistic function 
statements are inspired by the Guidelines t but the details of content 
come from the topics and vocabulary covered in the textbook. 

A perusal of the unit outlines for French I reveals that students 
are exposed to material beyond that which they are expected to master by 
the end of the course. For example, no ability in functional terms to 
handle the tenses other than the present is expected by the end of 
French I, yet the passe compose , imparfait , and futur are all introduced 
during the course. The instructional approach is cyclical; students are 
introduced to more advanced structures and practice manipulating them 
mechanically in structured drill activities; at a later stage, they will 
be exposed to readings containing these structures, and will be expected 
to use them in structured writing exercises, such as guided 
compositions. At an even later stage, they may be expected to use them 
easily and naturally in conversation and in writing. By this time their 
familiarity with the structures and theii general level of proficiency 
will have increased, making it possible for them to do so. This 
long-term, cyclical approach to the building of proficiency Is supported 
by leading methodologists. 

Miller and Cole plan to use the oral proficiency Interview primarily 
to evaluate their program, not to evaluate the proficiency of all 
students. The chief objection to the use of the oral interview in 
secondary schools is that it is Impractical. No teacher has enough time 
to administer interviews to all of his or her students. Miller and Cole 
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plan to assess students* oral proficiency as a tool to monitor their 
curriculum development project in the following ways: 

(1) interview a stratified sample of students at the beginning of 
French II and French II Honors to determine the proficiency 
levels of students who are beginning language study at the 
secondary level; 

(2) administer the oral interview to transfer students as an aid in 
placement; 

(3) interview all seniors and underclassmen who are terminating 
language study, in the hope eventually of recording the ratings 
on the students* transcripts; and 

(4) interview a random sample of students who have completed a 
two-year sequence, in order to monitor progress and assess the 
effectiveness of the program* 

The major innovation of Miller and Cole's work from a curriculum 
development standpoint is that their point of departure was the ACTFL 
Guidelines, an external set of performance goals that indicate 
directions for instruction* They then modified the textbook to suit 
their needs, creating their own materials as necessary to fill in the 
gaps* Most curriculum guidelines take as their point of departure the 
table of contents of the textbook, for two main reasons: (1) until the 
publication of the ACTFL Guidelines, there were no systematically 
articulated goals to allow for other than those implied by the sequence 
of topics and chapters in the publishers* text series; and (2) most 
teachers have not been able to devote the kind of time to the mammoth 
endeavor of restructuring a program that Miller and Cole have. 
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Case Study 2: Developing a Nev Curriculum for an Advanced Conversation 
at Northwestern University 

The general principle that performance assessment can lead to more 
realistic Instructional goals Is Illustrated in a project undertaken by 
Professor Isabelle Kaplan (1984) in a upper-division conversation 
course at Northwestern University. The course has traditionally been 
quite small — one section of approximately 15 students. Its clientele 
consists mainly of French majors and other students with a serious 
interest in developing oral facility in French who have not been able to 
study abroad* 

After becoming acquainted with the proficiency scale and being 
trained as an oral proficiency tester, Kaplan realized that she had 
pitched the course at an unreallstically high level. Most such 
conversation courses » serving students who have had three full years of 
college French, assume that students can intelligently and articulately 
discuss controversial and relatively abstract topics, such as US 
intervention in the affairs of Third World nations, or the issues 
surrounding local or national elections. When Kaplan conducted oral 
proficiency interviews of her students at the beginning of the semester, 
she found that all of them were at some point of the Intermediate 
level — speakers able to carry out short, simple conversations on 
familiar topics, such as home, family, and school. Clearly, world 
affairs and the pros and cons of prayer in the schools were far beyond 
their ability. 

The new conversation course that Kaplan developed was designed to 
move students from the Intermediate to the Advanced level, the level at 
which students can narrate and describe In past, present, and future 
time. She discovered in the process that the students did not know how 
to structure a narration or a description even in English. A large part 
of her planning, therefore, was devoted to activities that would develop 
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students' ability to tell a story with an informative beginning, an 
expanded middle, and a rounded-out conclusion. In the realm of 
description, students had to learn first to observe; Professor Kaplan 
developed a generic outline for all descriptions, listing for the 
students the elements of an object, place, or person that should be 
included. 

The course was designed on functional grounds — training the students 
to narrate and describe first in the present, then in the past, and 
finally in the future. Unlike most language courses, which are 
organized around a series of grammatical topics that are covered one at 
a time in linear fashion, Kaplan's course introduced gramnatical 
elements as they were needed to serve the functional goals. Students 
reviewed such grammatical topics as verb conjugations, noun-adjective 
agreement, and placement of pronouns as they were needed for the 
functional task at hand. 

The results of the course were two-fold: 

(1) Students' oral proficiency increased. When Kaplan tested the 
students at the end of the semester, she found that all of them 
were at the Advanced level. She credits the focused curriculum 
for the significant increase in proficiency; by knowing 
precisely where the students were, she was able to develop a 
course at the appropriate level. 

(2) The second result of the experiment was that enrollment in the 
course trip.led the next year. The students spread the word 
among their peers that "finally I learned how to speak French." 
They were correct; the Advanced level Is a very solid, 
serviceable level of oral skill. Although Kaplan did not 
develop her course as part of a formal research project with 
treatment and control groups, this spontaneous enthusiastic 
reaction to the course on the part of the students is evidence 
that significant and unusually rapid learning had taken place. 
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CONCLUSION 

In the final analysis, how can the ACTFL Guldellner. and the 
proficiency tests based on them work together to affect curriculum? 
There are several ways* 

First, and most important, projects like Miller and Cole's and 
Kaplan's utilize the Guidelines directly as the organizing principle to 
design a curriculum. Proficiency tests in both of these projects were 
used not to evaluate students' performance for purposes of assigning 
grades, but rather to generate data on students' level of proficiency as 
an aid in the curriculum development work* 

Second, tests in and of themselves drive instruct ion. Any kind of 
oral test will have the positive backwash effect of making students take 
oral production more seriously* 

Third, foreign language courses organized around functional goals 
are more effective* Students' efforts to learn a particular structure 
will not be wasted if the material is introduced at a point and in such 
a way that students are able to incorporate it into their growing base 
of .knowledge about and skill in the language. 

The "most basic, and perhaps most important goal that is inspiring 
the effort to promote proficiency-based foreign language education is 
that students emerge from language study with a usable level of language 
skill. The minimum level of skill that can be characterized as "usable" 
in this context is that needed to read a simple paragraph, ask and 
answer questions, and get from point A to point B in a country in which 
the language is spoken. Students who achieve this level of basic 
proficiency may well be motivated to learn more. Our hope is that by 
the time the next generation or two of students come to adulthood,, 
foreign language teachers will never again hear remarks at cocktail 
parties like "Oh, you're a foreign language teacher? I took four years 
of French in high school and I can't say a thing." 
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CURRICULUM GUIDELINES t TJIVRT. I 



SPEAKING 



Function 

Able to expreaa basic courtesy formulae. 
Able to make abort atatemente. 
Able to enumerate In short phraaea. 



Content 

' Immediate needs such as greetings, basic 
objects, days, months, colors, weather, 
etc. 



LISTENING 



Able to comprehend baalc courtesy, formulae, 

Able to comprehend basic vocabulary and 
memorized material. 

Able to comprehend utterances In highly 
oontttxtuallBed altuaticns. 



Immediate needs auoh as sreetlnga, basic 
objecta, montha, days, colors, weather, 
etc. 



Immediate needs auoh as in a restaurant, 
store, train, bin, etc* 



READING 



WRITING 

ERIC 
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Able to read baalc vocabulary and abort 
phraaea. 



Able to read Inatruotlonal and directional 
material. 

Able to read mastered material or recombina' 
tiona of maatored material. 



Immediate needa such as names, addresses, 
signs indicating names of streets and 
avenues, building names and short infor> 
mation signs. 
Immediate needs such as items on menus, schi 
dulses, timetables, maps and social codes 
(Defense de fumar, etc.). 
Basic survival needs such as messages, 
greetings and social amenities in dialogu« 
and specially prepared texts. 

Able to copy isolated words and short phraaes. Immediate needs such as name, address, dates 

numbers, days, months and common objects. 

Able to transcribe memorised words and phrases. Courtesy formulae such as cxpresalons in 

greetings and leave-takings. 

Abla to write memorised words, lists and ; 
phrases. 
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